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A SURVEY OF THE STATUS OF AND PHILOSOPHIES 


RELATING TO COCKPIT WARNING SYSTEMS 


George E. Cooper 
22701 Mt. Eden Rd . 
Saratoga, California 95070 


INTRODUCTION 


Background 

The development of the airplane has been marked by steady improvement in 
performance and mission capability. This improvement has been accompanied by 
more and more complex systems, the control of which requires that the flight 
crew be furnished with an ever-increasing number of cockpit instruments, 
controls, displays, and switches. 

The industry has responded to problems related to the management of 
these systems by providing a variety of warning and monitoring devices. In 
many cases a warning sound or light is introduced to alert the pilot to a 
situation that requires his attention and that otherwise might go un- 
recognized for some period of time. General standards have evolved in which 
a different sound is used for each audio warning. Often the louder or more 
distracting sound indicates the more serious condition; the use of color- 
coded lights (red to indicate the need for immediate action and yellow for 
delayed action) also categorize potential hazards and the appropriate 
responses. The general practice has been to add warning or cautionary 
devices one at a time as needs have been identified. This has led to a 
proliferation of warning sounds and lights which, in some cases, may have 
become counter-productive. . 


Statement of the Problem 

The ensuing piecemeal development of cockpit caution and warning (c/w) 
devices has come about with only isolated standards or guidelines being 
established for commercial aviation. Whatever standardization may have been 
achieved is probably due in part to the establishment of military specifica- 
tions, but the guidance they provide has little effect on the proliferation 
of c/w devices in the transport cockpit. Such standards as exist within the 
FAR’s are scattered and unrelated largely because they have been developed 
individually to meet apparent specific needs at the time and without 
consideration of overall cockpit design. The military specifications have 
gone somewhat further in consolidating basic standards for such cockpit 
systems and the human factors elements which relate to them, but cognizant 
military personnel feel that even these need revising. 



There is no way of knowing how many aircraft accidents have been 
prevented by existing c/w systems; certainly some of these systems have been 
at least partially effective in preventing accidents. On the other hand, 
there are a number of accidents which can be attributed directly or 
indirectly to the c/w systems themselves. A number of these, of course, are 
related to false or erroneous warnings, especially when these occur in a 
way that distracts the crew at a critical time, that is, when the crew*s 
complete attention should be devoted to managing the aircraft and making 
decisions. One example is the case of a false stall warning that occurred 
during takeoff. The warning sound was so loud that the crew could not 
communicate, and the result was an attempted abort with insufficient runway 
remaining . 

Cockpits of some modern transports have been compared to the interior of 
a cathedral because of the maze of colored lights required to convey the 
multiplicity of cautions and warnings. Two basic questions appear to be 
evident: First, is there any way to reverse the trend toward proliferation 

of warning and cautionary devices in the cockpit? Second, based on what we 
know today, are there any standards, guidelines or criteria that can be 
applied to the design of warning systems in future cockpits to improve their 
effectiveness? 


Program Objective 

The primary objective of the NASA program reported here was to take a 
broad look at current cockpit c/w systems, to examine industry philosophies 
regarding c/w system design, including current efforts to improve them, and 
to identify guidelines that are currently in use, delineating those which 
appear to have general acceptance, those which are considered ineffective or 
erroneous, and those with which there is broad disagreement as to their 
validity. The results of the survey are intended to provide guidance for 
NASA’s research and development programs in human factors. 


Method 

In order to accomplish these objectives, a broad industry survey was 
undertaken during which major airplane manufacturers — those concerned with 
large transports, general aviation, and military aircraft — were inter- 
viewed* Also included were several military and NASA establishments that 
have cognizance of cockpit design, research, and aviation safety. Finally, 
a manufacturer dealing specifically with aircraft instrumentation and 
electronics was consulted. A sunnnary of survey-related visits is shown 
below (a complete listing is provided in the Bibliography). 
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Transport aircraft manufacturers 

Domestic 4 

Foreign 3 

Military aircraft manufacturers 4 

General aviation manufacturers 2 

Instrument and supporting systems manufacturers 1 

Government agencies 

Regulatory, R & D 2 

Military Service, Safety, R 6f D 4 

NASA Centers, R & D 2 


The approach taken in arranging each visit was to inform the company of 
the purpose of the visit and to establish a mutually acceptable date. Upon 
arrival, the interviewer would explain the purpose of the visit, indicate 
that his interest was in the company's philosophy with respect to c/w 
systems, its specific views with respect to standards and guidelines they had 
found acceptable as well as those they had found unacceptable, and in 
defining areas of conflict where different solutions and opinions were 
evident. It was reasoned that as a result it should be possible to define 
specific questions and problems as well as possible approaches to their 
solution. It was pointed out that the purpose of the survey was not to 
enable NASA or the FAA to design a system for a future aircraft but rather to 
clarify problems and questions, solutions or answers to which were needed to 
guide NASA research and aid industry in designing more effective cockpits 
with respect to c/w systems. So that the opinions and recommendations 
obtained w6uld clearly represent those of * the persons being consulted, the 
interviewer made a specific effort not to introduce his own viewpoint into 
any of the conversations. It was also hoped there would be consensus with 
respect to some standards and guidelines, thereby establishing some overall 
acceptance and eliminating the need for research of some items. 

In each case the discussion focused on the equipment or technology 
available to the company in question and on those directions they felt it 
important to follow. A standard list of questions was not used, but in 
almost every case the discussions centered on audio, visual, and' tactile c/w 
devices; a ’^general" category was used for the items less specific in nature. 
This report discusses the subject by combining or contrasting the inputs from 
various organizations under each of these categories. 


RESULTS 


Tables 1, 2, and 3 list those guidelines or philosophies that were 
specifically mentioned during the survey program. Those mentioned most often 
in the separate interviews are listed first, with the remainder following in 
approximately decreasing order. Guidelines with general acceptance are given 
in table 1; general acceptance means that the guideline as stated was 
supported by the majority of those interviewed and that very little, if any, 
opposition was voiced. Table 2 lists those guidelines that were strongly 
supported by a minority, with few or no contrary opinions expressed by others; 


3 



this means only that these guidelines were not mentioned by the majority, 
not that there is no controversy associated with them. Table 3 lists those 
guidelines that received mixed reactions; they are listed separately to 
indicate that a marked difference of opinion exists. More than one-third of 
those interviewed expressed some opposing view. 


TABLE 1.- GUIDELINES WITH GENERAL ACCEPTANCE 


General 

1. '^Flight crews of current transport aircraft are overwarned." 

2. "Immediate action warnings should be kept to a minimum." 

3. "Both an audio warning and a visual indication of failure are generally 
required . " 

4. "Reliability of c/w systems is extremely important." 

"Warnings or signals that actuate too often are useless as c/w devices." 

5. "Standards or guidelines are needed for warning systems." 

6. "Warnings should be prioritized; some warnings should be inhibited 
during critical phases of flight." • 

7. "New Warning system additions to the cockpit should be thoroughly 
evaluated before being placed in service." 

"Full mission simulation should be used to study the human factors." 

8. "It should be possible to cancel most warnings." 

9. "Electronic display, computer logic, and checklist monitoring are 
desirable advances, as is the positive trend toward computers and more 
automatic systems." 

iO. "Air crew workload is often too high." 

"The major air crew task is to fly the airplane" (not to operate 
systems) . 

"More flight engineer duties and aircraft systems should be automated." 

Auditory 

1. "Audio warnings or signals should be limited to four or five." 

2. "Continued loud sounds tend to incapacitate." 

3. "Voice warnings are desirable." 

"It is acceptable for all audio warnings to come from a single source 
and to be electronically generated." 
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TABLE 1.- Concluded. 


Visual 

1. ”A central warning system is needed in the cockpit." 

"All visual warnings should be concentrated within a 30° visual cone," 
"Anything displayed on the central warning or caution panel must also be 
displayed somewhere else." 

2. "A darkened cockpit is favored for normal operations." 

3. "Lighting intensity and contrast are a serious problem in c/w systems." 


TABLE 2.- GUIDELINES RECOMMENDED BY A FEW BUT OVERLOOKED BY MOST 


General 

1. "Configuration warning is necessary for certain flight phases." 

2. "Most urgent warnings should be related to the control involved." 

3. "A requirement exists for additional warnings, that is, attitude, 
turbulence, wind shear and collision avoidance." 

Auditory 

1. "Audio warnings should be used for pilot error situations only." 

2. "A radio override switch is needed to reduce interference by c/w systems." 

3. "Voice warnings should be advisory in nature." 

Visual 

1. "The value of large, easy-to-read lettering or messages in a c/w system 
has been under-emphasized." 

2. "A third color in addition to red and yellow is needed in caution/warning 
systems . " 

"The use of all colored lights in the cockpit should be standardized." 
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TABLE 3.- GUIDELINES WITH MIXED REACTION AND ACCEPTANCE 


General 

1. “An improved c/w system for retrofit is desirable. “ 

2. “Reverse the trend toward teaching pilots basic systems only (BSO) , that 
is, procedures only without a thorough knowledge of the aircraft systems." 

Visual 

1. “A master warning light should be used." 

2. "The annunciator panel should display only system malfunctions and not 
pilot errors." 

3. "Flashing lights should be used in c/w systems." 

"Flashing lights are excellent attention-getters but are seriously 
distracting. " 

"Variation in flash rate should be used to indicate criticality." 

Tactile 

1. "The use of tactile warnings should be extended in order to diversify 
the inputs to the pilot." 


DISCUSSION 


In the following sections, the guidelines and philosophies are used as 
the basis on which discussion is conducted under the subheadings of general, 
audio, visual,, and tactile. Where helpful in illustrating a point, actual 
examples drawn from the interviews are also presented. Recommendations for 
research or future study derive from the interviews. 

Comments or conclusions expressed in this section of the report are 
generally from the persons interviewed, unless identified otherwise. In the 
following Analysis and Comment section conclusions of the author are also 
included. 


Generally Accepted Guidelines; General 

1. " Flight crews of current transport aircraft are overvarned ." There 
was almost universal agreement with this statement. The only exception 
found was with the manufacturers of the smaller general aviation aircraft 
which so far have escaped the proliferation of warning and alerting lights 
and sounds. These manufacturers, however, did recognize the beginning of a 
trend in this direction. They were more concerned, however, about 
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increasing regulation, which requires more and more instrumentation and 
special equipment to operate in today's airspace. There were few who thought 
more warning devices were needed. It was apparent, however, that they were 
anticipating future development for collision avoidance, turbulence, 
altitude, and windshear warnings. This comment was therefore interpreted to 
mean that new requirements for warning and alerting devices may be required 
but that in the overall picture the total number must somehow be reduced 
and/or incorporated in the cockpit with more concern for the human factors 
involved. One manufacturer felt that we could be creating more hazards than 
solutions by the addition of more and more warning and alerting devices. 
Another opinion expressed was that aircrew members are becoming warning- 
dependent because of the trend toward providing a warning for every possible 
pilot transgression or error. One manufacturer indicated that this 
appeared to be associated with the trend toward teaching pilots the basic 
systems only, that is, teaching procedures without a thorough knowledge of 
the aircraft systems, and that this was a fundamental error. This subject 
is further discussed later in the section on Mixed Reaction and Acceptance. 

2- '^ Immediate action warnings should be kept to a minimum ." The 
implication here was that too many immediate action warnings will render 
immediate response impossible or force pilot action that could be in error. 

A maximum of 10 was referred to. Most comments were not specific but 
endorsed the concept. While it was not always stated in the same words, 
there was a desire expressed to see a drastic reduction in the number of red 
lights or other immediate action warnings. With one transport aircraft, the 
red and yellow annunciator panels were separated, and each pilot had his 
own red emergency warning panel located at the side. It was necessary for 
both the pilot and copilot therefore to look to the left or right side panel 
to ascertain the problem after first being alerted by his own centrally- 
located master warning light. The large number of red lights in this panel 
was considered quite undesirable. 

One Air Force spokesman would like to see the large amount of prime 
instrument panel space now devoted to c/w lights reduced. The trend toward 
the master warning light and central panel (with up to 100 lights) is 
dictated by a requirement for some rational organization which can be better 
managed by the crew. 

3 - " Both an audio warning and a visual indication of failure are 
generally required ." It was generally agreed that more than one channel of 
input should be used to assure reception of a warning by the crew. The most 
commonly used combination is audio plus visual, but in this context a 
tacile warning could be used with either visual or audio in certain cases. 

One concept advanced for the use of multiple warnings was that the audio 
signal alerts and the visual signal identifies the problem. There appears to 
be merit in this concept, but there are others who recommend the use of 
flashing lights rather than audio signals for alerting the pilot. In the 
author's opinion, there also appears to be merit in considering the alerting 
function as separate and distinct from the information and action function 

of a c/w system, but this will be discussed further in the Analysis section. 
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4 . 


^^ Reliability of c/w systems is extremely important . '' 

*^ Warnings or signals that actuate too often are useless as c/w 
devices." Nuisance warnings, whether caused by unreliable systems or by 
design error, contribute to a pilot’s ignoring an indication when it is a 
real one. The example mentioned most often in this regard relates to the 
altitude alert, wherein, the alert signal, both light and tone, are excited 
as much as 1000 to 1200 ft prior to reaching the set altitude. This was 
considered an objectional feature by a great majority of those interviewed 
and considered desirable by only a very small percentage of pilots. If a 
warning sounds too often, it was pointed out, the pilot develops the habit 
of "punching it out" without thinking, and it loses its value. A few pilots 
learn to depend on it for preventing altitude overshoot so that failure to 
operate can be more serious- An auto-pilot disconnect, which actuates an 
audio warning on each frequently occurring disconnect also was noted as 
losing its value. True reliability, therefore, includes the human element 
as well as reliability of hardware. 

One of the simpler methods of ensuring reliability in the human element 
lies in not having systems that "cry wolf." Such is the case when the crew 
is forced repeatedly to punch out a light or silence a warning sound or dis- 
regard nuisance indicators. Systems that have been so identified include 
the altitude alert, the GPWS during its initial period of operation, and 
lights that flash during engine start or normal system operation. A simple 
solution for improving the latter has been obtained by introducing a short 
time delay (50 to 75 msec) in the indicating lights of a master caution or 
warning panel. This eliminates spurious lights flashing due to aircraft 
acceleration or momentary switching of electrical systems. One company which 
employs this method Indicated that a somewhat longer time was required in 
the case of low oil pressure indications. In spite of widespread criticism 
of the GPWS , one manufacturer has documented numerous instances where the 
device has not only prevented potential accidents but has also detected 
instrument approaches that were marginal with respect to initial approach 
terrain clearance. Causes of nuisance warnings have been determined and 
corrective action taken such that they are now reported by one GPWS manu- 
facturer to be almost nonexistent (ref. 1). 

Avionics reliability was generally praised and considerable credit given 
to the RTCA committee which established the criteria. The current practice 
of packaging cockpit instruments and avionics separately and developing each 
cockpit from the desired "modules" has certain reliability benefits, in that 
failure of one subsystem affects only that element. The additions of 
altitude alerting and the GPWS are recent examples. The modular concept, 
however, does have certain disadvantages. One is the adverse effects on 
standardization. One company pointed out that It must be prepared to install 
any one of 18 different ADI’s in its aircraft, depending on customer choice. 
Possible adverse effects can also occur with c/w systems in that simultaneous 
operation of several warnings can result in delayed response by the crew 
because they must first sort out the various indications and formulate 
decisions before taking action. This is in conflict with the strong 
recommendation for some method of prioritizing warnings. The Air Force now 
recognizes the need for total system analysis if overall system reliability 
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is to be obtained. Such a system reliability study is currently being 
conducted for the Category III landing. 

The high reliability achieved with avionic systems to date was given 
high praise and specifically credited with the current success of the head~up 
display, which combines and integrates a variety of information, that is now 
in use by the military. One problem which apparently has not been completely 
solved is how to remove unreliable information from a HUD, The central air 
data computer (CADC) was noted as a system that has become extremely effective 
because of its high reliability. One fighter aircraft has been flying nearly 
4 years without a single failure in its prototype. 

Some of the simple systems mentioned for improving reliability are the 
use of dual lamps in c/w panels for redundancy, the use of a systems* test 
panel wherein the systems can be tested on the ground or during preflight, 
and fail-safe circuitry design utilizing a logic system that provides 
detection, through a test switch, of a broken wire or sensor problem. 

New, sophisticated aircraft also contain an additional system for 
improving overall reliability. One such system is referred to as the BITS 
(built-in-test-system) , which enables an operator to trace faults throughout 
a system and to isolate causes of specific failures that may be indicated on 
an annunciator panel. 

The cause of a large family of nuisance warnings was illustrated during 
discussions regarding space systems with astronauts and other personnel. 

This was a particular problem during the Apollo program wherein many warnings 
and cautionary indications plagued the operation because the limits on many 
systems were set too tight. As a result, as each system went out of limits, 
a warning would occur. In a strict sense, then, most Apollo c/w*s actually 
were out-of-limit indications, nuisance warnings, rather than true faults. 

This problem was noted to have two possible solutions: (1) To employ 

a built-in test system wherein an operator can interrogate in minute detail 
the various elements of a system to determine where the fault lies, and (2) 
after using the test system to determine when a fault is merely out-of-limits 
a small amount, a capability is provided allowing in-flight adjustments to 
the system to reset limits and thereby reduce nuisance warnings. The 
astronauts with whom this was discussed expressed some concern about making 
these actual adjustments in flight for fear of "fouling up" a system. A 
conflict therefore exists between continued acceptance of such nuisance 
warnings or acceptance of the risk involved in making in-flight adjustments 
to established system limits. 

The space shuttle will use a special status panel to isolate faults and 
determine the out-of-limits condition. It was pointed out that a dedicated 
computer was required for such a software system and that the computer must 
have a very large capacity for such a complex task. With respect to 
redundancy in display elements, a triple CRT display will be available 
wherein any c/w message can be shown on any one of three displays. 
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The conflict between the continued use of separate, modular systems in 
the cockpit versus combining these into a single unit and accepting 
automatic prioritizing has been mentioned. Reliability of the single voice 
warning unit which contains multiple warnings with automatic prioritizing 
was reported to be very high, partly because many internal failures can occur 
before readability of the message is lost. 

5. **S tandards or guidelines are needed for warning systems. ^' Through- 
out the sur^^ey, it was apparent that military standards regarding cockpit 
design and human factors considerations (Mil. Stds. 1472 and 411D, Aircrew 
Station Signals, Human Engineering Design Document) have had a significant 
effect on the development of many warning systems in both civil and military 
aircraft. Of those commenting, practically all specifically favored the 
development of greater standardization. A few, including manufacturers of 
general aviation aircraft, were concerned that this might lead to additional 
regulatory requirements. A surprisingly high percentage of comments favored 
the adoption of standards or guidelines for cockpit warning and alerting 
devices for civil application. While no detailed comments were obtained nor 
study made of the FAA standards, the feedback obtained indicated that 
Federal Air Regulations (FAR) pertaining to cockpit warning and alerting 
were widely scattered and more restrictive than helpful. Nevertheless, one 
transport manufacturer suggested that the FAA should take the lead in 
achieving greater standardization in this area for civil aircraft. 

It was apparent that military specifications are not applied uniformly 
by all military contractors. This suggests that the military specifications 
function effectively as guidelines to effect a degree of standardization 
without seriously impairing new development. It appears probable to the 
author that this could also reflect differing viewpoints of system project 
officers (SPp’s) or else the lack of sufficient justification being 
presented by the manufacturer for a waiver or deviation. Air Force personnel 
confirmed that the human factors document, Mil. Stds, 1472 (ref. 2) was a 
very general one and did actually serve primarily as a set of guidelines for 
the designer. The Air Force also indicated that there was a current effort 
under way to revise those parts of Mil, Stds, 411D relating to "Aircrew 
Station Signals" (ref. 3). The current effort recognizes the need to include 
the rationale behind the specification. 

The Boeing Company, under an FAA contract, has done considerable work 
in defining requirements for an independent altitude monitor. Included in 
this study is a very extensive literature search that compiles a great deal 
of basic human factors data that is fundamental to cockpit design from a 
man-machine integration point of view and could assist in development of 
standards for civil aircraft. It includes such items as stimuli response 
data, alerting philosophy, and concepts (ref. 4). 

An effort is currently under way by the S-7 Committee of the Society 
of Automotive Engineers not only to revise SAE Standards for civil aircraft 
but also to provide design guidelines for improved future c/w systems 
(ref. 5). There would appear to be distinct advantages in maintaining some 
level of coordination between the S-7 Committee effort and the military. 
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Currently, this occurs through common civil-military contractors. While 
identical standards are probably not obtainable and may not even be 
desirable, there are many areas where, because of basic human factors 
involved, there will be agreement with respect to guidelines and criteria. 

6. ^’ Warnings should be prioritized; some warnings should be inhibited 
during critical phases of flight/ ' Both of these guidelines also relate to 
the reduction of emergency warnings to less than 10. There was general 
agreement that an individual crew member can handle one emergency at a time 
and that additional extra warnings distract him from the task at hand- As a 
result, there was unanimity of opinion that it should be possible to cancel 
most warnings. Some means, however, must often be retained in certain 
situations for decision-making and prioritizing by the crew. 

On a number of new aircraft, such as the Concorde and A-300, a takeoff 
inhibit mode is provided where the pilot may cut out all but a few critical 
warnings during the takeoff phase. Such a system reduces the potential 
distraction during a critical period and eliminates the possibility of 
aborting the takeoff without sufficient reason. On one foreign aircraft, 
fire warning is actually inhibited during the takeoff phase. One con- 
tributing factor has been the large number of false fire warnings. Another 
was concern about the possibility of shutting down the wrong engine, which 
has been and often still is a possibility. Even conducting cleanup checks 
on the engine after a shutdown has a certain potential for affecting the 
wrong engine. Finally, it was concluded that an engine fire can usually be 
better handled in the air than on the ground during an abort. This remains 
a controversial subject, however. 

Inhibit capability is, in reality, a form of prioritizing, and the 
ability to cancel certain warnings is a manual means of inhibiting. While 
there was some concern about carrying prioritizing too far, the capability 
of inhibiting during certain mission phases appeared to be universally 
acceptable. The concern relates to the fact that the pilot should always 
retain the final authority for establishing priorities. With multiple 
failures, however, it would appear desirable that he receive additional help. 
An extreme example of this would be the case involving Apollo 13 when an 
emergency caused by the explosion of an oxygen bottle in the command module 
created a ’’domino’* situation which resulted in the entire panel being alight 
with warning and caution lights. The complex effects of this emergency 
could be sorted out only by use of a team of experts working with the backup 
vehicle as a simulator back on earth. By careful simulation, determining 
causes and effects, the scientists, engineers, and technicians on the ground 
were able to determine the extent of the damage, what systems were affected, 
and the risks involved either in continuing flight or returning to Earth as 
soon as possible. The point is that it was not possible to deduce from all 
the lights exactly what proper emergency action should be. 

Once inhibiting is accepted as a means of reducing the number of 
warnings the pilot is subject to, there are several ways open for its 
implementation. One is through manual inhibiting by flight phases, relying 
on crew action for activation. The other is by use of a computer which uses 
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preplanned logic that will respond to an order of priorities in presenting 
warnings as well as determining the level of warning that needs to be 
displayed. By this it is meant that failure of a first system might result 
in a caution, whereas failure of a second system would result in an emergency 
warning. This concept has been carried forward even further with a proposal 
called PAWS (Phase Adapted Warning System) (ref. 6). In this case a switching 
logic module receives information from the central annunciator panel and some 
other sensors (i.e., airspeed, altitude, etc.), It then can produce one of 
three possible outputs to the warning lights: red for immediate action, 

yellow meaning "caution or delayed action," or a third signifying "hold," 
but without any audio or visual display or action signified, 

7 . " New warning system additions to the cockpit should be thoroughly 
evaluated before being placed in service . " 

" Full mission simulation should be used to study the human factors . " 

Due to evolutionary development and piecemeal additions of c/w items, 
insufficient attention has been devoted to the human factors aspects of the 
resulting cockpit in terms of overall workload requirements and pilot 
acceptance of the c/w system. While testing of some individual components 
may have been extensive, the resulting combinations have been subjected only 
to the operational evaluation obtained from actual line or service use. The 
majority of those interviewed expressed the need for a more thorough evalua- 
tion of proposed changes to cockpit warning and alerting systems before 
placing them in service. 

^ It was felt that if this had been done, many of today's problems 
would have been identified and analyzed much earlier, thereby enabling either 
correction or discontinuation of unsuitable c/w methods. In addition, a 
degree of testing in the operational environment is usually required to 
ensure system reliability and elimination of most nuisance warnings. 

Although the comments received differed in form, they supported, in 
general, this guideline. Some were more specific in stating that such an 
evaluation program must be objective in nature and should be conducted by 
an unbiased group. 

It must be stated, however, that there was evidence of considerable 
testing being done by manufacturers, military agencies, and others to guide 
specific cockpit designs and acceptance of new technology. The cases in 
which these efforts failed or were incomplete seem to be related to the lack 
of the total workload concept in the evaluations, such as would be provided 
by full mission simulation. Recent studies have shown that such methods do 
have the capability of providing realistic workloads and, more important, 
of eliciting the same actions, reactions, and decisions from the flight crew 
as the actual flight task would. 

8. ” lt should be possible to cancel most warnings ." This guideline 
stems also from the proliferation of warnings and applies to both the visual 
and audio systems. Any device which is sufficiently attention-getting to 
alert a crew member also has the potential for creating a highly distractive 
environment. This conflict appears to many to be resolvable only by 
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providing the pilot with a means of cancelling the warning signal once it has 
accomplished its primary purpose of alerting. Some of those interviewed were 
concerned that a warning which cannot be cancelled except by correction of 
the fault has a tendency to force the pilot into precipitous action which 
might be erroneous. Extremely loud or visually distracting alerting systems 
can interfere with cockpit communication, decision making, and crew 
coordination. In such cases, cancellation of the warning becomes essential 
prior to taking corrective action. 

9- " Electronic display, computer logic, and checklist monitoring are 
desirable advances, as is the positive trend toward computers and more 
automatic systems. " These guidelines will be discussed together, l^ile these 
trends are recognized and appear acceptable to the industry, there is some 
concern about what the limits to automation and sophisticated computers 
really are. 

Recent work by Boeing, known as the Automatic Systems Monitor (ASM) 
program, in which CRT's were utilized in a 737 simulator study to explore the 
replacement of basic engine instruments and the application to providing 
better checklists, is very encouraging. This program also made use of full 
mission simulation as an evaluation technique. The results of the Boeing 
work also appear to have influenced the design of the space shuttle cockpit, 
in that three CRT's are being used there as a means of improving the 
information display for the astronauts. The astronauts who evaluated the 
Boeing program were particularly impressed by the way information was split 
on the CRT, providing not only basic warning information in the upper half 
but also displaying, in the lower part of the display, the action to be 
taken. Another concept employed in the Boeing ASM program included the 
provision for displaying the layout of any given system, together with 
associated procedures, in response to an inquiry by a pilot. It was 
emphasized that much work is still required on how best to display the 
information so that the pilot can rapidly assess the problem and appraise 
what he is doing. The mere lifting of tables and instructions from manuals 
is not considered to be an acceptable solution. 

The use of computers will be vital to future c/w systems if such 
apparently desirable features as prioritizing and inhibiting by phase of 
flight are adopted. It was also pointed out that the big advantage in the 
use of digital equipment lies in its self-monitoring capability, available 
without additional equipment. 

In modern, sophisticated vehicles such as the space shuttle, a 
limitation in computer capacity can be encountered which places practical 
limits on the extension of caution and warning systems to handle everything 
considered desirable. Complex computer systems have been employed on recent 
aircraft, such as C-5 and B-1, to enable isolation of faults in response to 
c/w indications. While problems have been encountered in the use of such 
complex systems during operational phases, indications are that, given 
sufficient support, computers will play an important part in enabling 
on-board isolation of the detailed causes of c/w indications. Central 
integrated test systems, for example, are reported to be extremely effective 
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and reliable in this regard if properly maintained. Such systems become more 
important as the operation of more and more aircraft systems is conducted 
automatically, leaving fewer indications to the crew, other than that a fault 
exists within a given system. 

10. * *Air crew workload is often too high .'* 

The major air crew task is to fly the airplane ^* (not to operate 

systems). 

** More flight engineer duties and aircraft systems should be automated . ” 
The significance of the high aircrew workload under certain flight conditions 
and its effect on crew-member response to the c/w systems is illustrated by 
the following example. During takeoff, a senior pilot was informed that an 
engine would be cut on him at some point for his evaluation of handling 
characteristics and safety. During the takeoff an actual fire warning 
occurred, but the pilot was so intent on handling the engine out that he 
failed to recognize the engine fire, even though all warnings were provided. 

Other examples were noted in which pilots failed to hear warning sounds or to 
observe warning lights under conditions of high stress and concentration. 

This, of course, was a primary reason for the recommendation that multiple 
channels be used to provide c/w signals. The conclusion is evident, therefore, 
that under high stress and workload conditions there is probably no way that 
one can be assured of a warning being recognized and acted on properly by a 
pilot. Crew coordination and monitoring procedures, therefore, assume a much 
greater importance. There can be no doubt that this fact has indirectly led 
to some of the existing problems, such as the use of many loud audio tones 
arid sounds, and centrally located flashing or steady red and amber lights. 

All were designed to insure catching the pilot’s eye or ear, but primarily 
causing a distraction and adding to the workload. 


Generally Accepted Guidelines: Auditory 

1. ” Audio warnings or signals should be limited to four or five .” Of 

all recommendations made by those interviewed, this one was mentioned most 
often. One concludes that there is universal consensus that audio warnings 
must be reduced in total number and in types of sounds. 

It was not uncommon for aircraft to have 10 to 12 different audio 
sounds, and in some aircraft the number of tones exceeded 30 or 40. There 
is little wonder, then, that concern was expressed about identifying a given 
tone with a given warning. It was pointed out that some sounds might be 
reduced by inhibiting them except for a critical flight regime for which 
the warning was needed. Because of the many audio signals in use, it was 
recommended by some that preflight familiarization with them was essential. 
Others were not in favor of this, believing that this was more distractive 
than helpful, and as a solution recommended the overall reduction of sounds. 

Military Specif ications provide no standards in this regard. As a ^ 
result, innumerable pilot evaluation programs have been conducted to 
determine which sounds will be vised on each aircraft. The result has been 
a proliferation in number as well as in different types of signals 
throughout the industry. 
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One solution that has been incorporated in some recent foreign 
transports has been the use of a single alerting tone (gong) for either 
all warnings and cautions or just warnings. The only objection found to 
this was the fact that many feel it important to retain, because of their 
long history of use, the bell and horn used for fire and landing gear 
warnings. 

It was pointed out that a reduction in the number of audio warnings 
could be effected by eliminating all audio signals from cautions or lower 
level advisories. This, however, remains controversial. Even if a single 
alerting sound is used, there was a question of whether it should be steady, 
intermittent, or automatically cancelled after a given period. 

There was a plea from a significant number of those interviewed for 
development of some standards in this regard; it was recognized that some 
additional objective evaluation program might be necessary. 

2. ** Continued loud sounds tend to incapacitate .*’ It was pointed out 
that, if a pilot is unable to cancel or diminish a loud alerting sound 
except by correction of the fault, he is very apt to be pushed into 
precipitous action, thereby inadvertently taking incorrect action. 

This philosophy of continued loud warning sounds was frequently 
emphasized and has led to certain recommendations. One is that the pilot 
should be able to inhibit or cancel any audio tone. Another is that the 
pilot should have control over the intensity of the sounds. Both of these 
have been factors in known aircraft accidents. Current military standards 
require that any warning sound must have a sound level higher than the 
maximum ambient noise at the warning frequency. While this has a certain 
rationale, it could require sound levels that are highly distractive and 
which could incapacitate the crew. VJhile the ability to cancel a warning 
has already been discussed as highly desirable, the provision of an 
intermittent signal, for example on 10 sec, off 10 sec, was recommended as a 
solution in some cases, since it would at least provide brief periods of 
freedom from the distractive sound during which coherent thought could be 
resumed. Recommendations were also received that a 10 sec audio signal 
should be sufficient for alerting purposes and that it should be auto- 
matically cancelled after this or some other experimentally determined 
interval. Conflicting opinions were also received with respect to an audio 
signal for autopilot disconnect, which often was considered a nuisance. 

This was partially resolved, however, by the difference in consequence and 
hazard associated with such a disconnect on different aircraft or flight 
regimes . 


3. '’ Voice warnings are desirable .** 

" It is acceptable for all audio warnings to come from a single 
source and to be electronically generated . " 

The problems associated with having too many auditory signals no doubt 
enhances the desirability of verbal warnings. The problem of associating 
various sounds in audio signals with a particular failure is eliminated if 
a voice message is used to tell the pilot clearly what has happened. The 
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Air Force indicated that research has shown that there is a 6 to 9 sec im- 
provement in recognition through the use of such direct cues, a concept 
which they favor highly. There were essentially no objections to the concept 
of voice warning even though many of those interviewed had no direct 
experience . 

One statement was that pilots want and need more information through 
the auditory channel. Many pilots themselves concluded that voice is the 
way to go because a message clearly stated reduces acceptance time, even 
though they would also want a backup visual identification. Air Force pilots 
indicate that their heavy reliance on a visual c/w system is simply the 
result of having too many tones to identify and relate to. Therefore, the 
direct, simple solution is the use of "plain English" in the c/w system. 

There were no recommendations for the use of voice warning except in con- 
junction with some type of visual system. There were few limitations placed 
on the use of verbal messages in a c/w system except insofar as auditory 
signals were generally recommended for warnings rather than cautions. 

Military standards actually limit the use of voice to warnings only and 
require a separate alerting tone to precede it. 

One of the concerns with respect to voice warnings was why, after the 
initial research and applications in a B-58 years ago, it had never been 
implemented. In talking with B-58 pilots who had experience with the system, 
no real objection could be found except that it was a tape-driven system, 
somewhat unreliable due to development problems initially, but that it had 
done a good job in supplementing the information from a poorly located 
caution panel. This panel was located to the left and rear of the pilot 
and could be seen only by turning the head and looking low and to the rear. 
One report was that there was a strong tendency to make use of the voice 
information with the B-58 and to ignore the visual, which is not too 
difficult to understand considering the location of the annunciator panel. 

One study conducted by the Air Force was reported to have shown that 
98% of those interrogated favored voice warning, based on their B-58 
experience. This is in spite of the fact that this taped system was 
considered an inferior one and was earlier fraught with reliability problems. 

The primary problem of introducing voice in military aircraft appears 
to be one of paying the added cost of equipping a prototype with a voice 
system without having data on the benefits. So far, the simplest approach 
has been to go along with the standard audio-visual warning system composed 
of sounds and lights. At least two recent Air Force aircraft programs 
included plans for a voice warning system on a prototype, but both had been 
cancelled to conserve funds. Voice warnings have been used quite 
effectively in missile crew work. One difference was noted, however: the 

conditions surrounding a missile launch represent a more static rather than 
dynamic situation such as that present during aircraft operation. 

There were a number of researchable questions raised involving the 
application of voice warning. Military Standard 411D specifies verbal- 
warnings for immediate action items only, whereas some responses anticipate 
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a broader application. One concern is the possible interference with or by 
ATC, other radio comniunications , or from internal cockpit communications. 

The earlier proposals to use a feminine voice, which would be more readily 
recognized and discernible, is no longer valid because of the increasing use 
of women in control towers and ATC. Electronically generated voices, 
however, have the potential for being altered through a frequency tone makeup 
which could be quite different and therefore more readily recognized from 
the various communications systems. Some concern still exists about the use 
of proper terminology in voice warnings. The possible differences in 
interpretation of certain words and phrases by different individuals, 
particularly by those of different nationalities, were considered to indicate 
that continuing tests should be conducted until such points are resolved. 

One concern expressed was that an adequate voice warning system would require 
a large computer prioritized by phase of flight. This is somewhat in 
conflict with other opinions which indicate that a small, compact, 
electrically generated voice system would be extremely easy to retrofit and 
would have sufficient capability for current transport aircraft. 

A serious problem facing the use of a single electronic generator for 
voice warning appeared to be inertia in the aviation system that must be 
overcome to enable acceptance of such a change. The need for prioritizing 
also appears to be very important if the full capability of electronic voice 
is to be achieved. Currently there are a number of impediments to the 
adoption of prioritizing. One is the present trend toward the modular 
development of electronics in separately packaged boxes which are not as 
conducive to prioritizing and could allow simultaneous warnings to be 
sounded. Assuming as fact the conclusion that a pilot can handle only one 
warning at a time and must ultimately resort to his own prioritizing in the 
case of multiple failures, the majority foresaw no serious problem with 
prioritizing as long as it is done sensibly and the pilot is informed that 
additional warnings await recognition. The second impediment is the conflict 
with some existing regulatory requirements dealing with audio signals, 
including the use of voice, which require resolution. General opinion has 
been seen to favor a dual audio-visual warning system. A few of those who 
were most emphatically in favor of voice warning felt that such a system 
could stand by itself without the visual backup. This is a researchable 
question requiring justification. 

With respect to pilot acceptance, there was strong sentiment expressed 
that voice warnings should be advisory in nature rather than commanding. 
Pilots almost universally appreciated advisories such as **Glide slope, glide 
slope,'' indicating an out-of-limits deviation, but considerable objection 
was raised to the command, "Pull up, pull up." This is discussed more fully 
under the specific guideline on the subject. 

In summary, voice warning was recommended strongly as offering a great 
potential for improving cockpit warning systems, either through retrofit or 
as part of an advanced system. Unless prioritizing as a concept can be 
established, however, much of the benefit potentially available from 
electronic voice generators may not be realized. While some experimental 
work has been completed giving insight into the basic elements of voice 
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messages, an evaluation study of voice warning, incorporating varied 
terminology and including other audio signals, both advisory and of a 
command nature, was recommended. 


Generally Accepted Guidelines: Visual 

!• central warning system is needed in the cockpit .** 

* *A11 visual warnings should be concentrated within a 30"^ visual cone . 
‘^ Anything displayed on the central warning or caution panel must 
also be displayed somewhere else . " 

These statements reflect the necessity for ensuring that visual warnings do 
not go undetected. The current trend is to place all red warning lights 
within a 30® cone of vision and to centralize visual caution signals within 
a central annunciator panel even if this does not lie within the 30® cone. 

In concept this appears rational except for the proliferation of lights and 
the need to extend beyond the 30® cone in some cases because of central panel 
space limitations. This leads in turn to a requirement for master warning 
lights, either red or yellow, or both. The competition for central panel 
space is so severe that little or no consistency has been achieved in either 
the location of warning lights or of annunciator panels. The result has 
been that most red warning lights are scattered about the forward panel, 
generally within the visual cone, but separated with some on the glareshield, 
some on the panel, some on either side of the console. This is also true on 
fighter aircraft where landing gear warnings are often well outside this 
cone. 


The basic visual warning system philosophy using colored lights follows 
the arrangement described as follows: Primary alerting is obtained through 

red and/or amber master lights located within the 30° cone, preferably near 
each ADI, as a means of catching both the pilot's and copilot's attention. 
The military specifications, however, require they not be within the basic 
group of flight instruments. If a master yellow light is used, it merely 
directs attention to the primary annunciator panel, the lights on which are 
sometimes referred to as "director” lights because they direct attention to 
other yellow lights, usually outside the 30® cone. These third level 
yellow lights are located, as' a rule, with the particular system or control 
to which they relate. In this manner, the pilot's attention is ultimately 
directed to the specific system which has failed. Supposedly, he will know 
where this panel is located even though it may be to the rear or on a flight 
engineer's panel. Arrows are sometimes used to indicate general location 
of the system in question. These are often located in places difficult to 
reach or otherwise inaccessible to normal vision. 

Additional red warning lights are positioned wherever space can be 
found on the forward panel or glareshield. As additional emergency warnings 
are added, more red lights are necessary. On military fighters, the rim of 
the glareshield is normally used. Panels used for c/w*s take on a variety 
of forms. On some the red and yellow lights are separated by grouping; on 
others they are mixed, and in some cases separate red and yellow panels are 
used. If a central location cannot be obtained, quite often either the red 
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or amber panel will be duplicated so that both pilot and copilot have one. 

A master warning light becomes necessary when some warning lights cannot 
be centrally located within easy view of either pilot. 

In general, the system of warning lights described here, when properly 
implemented, were noted to be "not too bad," "reasonable," or "acceptable." 
Properly implemented refers to readable labels or messages on the lights, 
red emergency lights well located for each pilot, and central annunciator 
panel also easily viewed by both pilots. This system allows caution and 
warning systems involving large numbers of lights (30-100) to at least 
become reasonably manageable. 

In order to achieve compliance with the 30° visual cone, it has become 
necessary on most military aircraft to mix red and amber lights in the 
central zone. Examination of these c/w lights, which are clustered about 
the ADI, or at least within this 30° cone, reveals that, with the exception 
of a master caution light, they relate, on the newer aircraft, primarily to 
pilot items, that is, pilot failure or error items. This mixture would 
appear to reflect a guideline suggested by some to the effect that "Only 
systems malfunctions should be shown on the annunciator panel." While the 
latest labeled-light concepts employed on the most recent military aircraft 
were considered generally acceptable, steps for improvement were still 
requested . 

"Anything displayed on the central warning or caution panel must also 
be displayed somewhere else," was a philosophy also stated in the reverse, 
which says that any caution or warning displayed anywhere in the cockpit 
should also be shown on the corresponding central panel. For example, a 
master warning/caution light might indicate by color a caution which directs 
attention to a central annunciator panel. Here the system in question 
(i.e., electric, hydraulic, etc.) is identified by an illuminated, labeled 
light. Reference is then made to the electrical panel where it becomes 
apparent that No. 2 generator, for example, is out, and where action, if 
necessary, is to be taken. 

In the case of warnings, it was obviously desirable to reduce the 
total number of lights involved; for example, in case of fire, a red light 
comes on in the actual fire handle, which at once identifies the particular 
engine on fire and tells the pilot where his fire suppression handle is. 

If not incorporated in the fire handle, good practice calls for illuminating 
the particular engine fuel shutoff valve which may be located separate from 
the fire handle. This precludes shutting off the fuel to the wrong engine. 

What has not been considered here is the case where all information for 
alerting, identification, and action may be provided on a central panel — 
a situation usually unattainable because of panel space limitations — or 
that requires a computer-generated display. 

2* " A darkened cockpit is favored for normal operations. ** A cockpit 
which is alight with any c/w indication during normal operation or without 
cause is generally considered distracting and detracts from a pilot’s ability 
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to detect change. Several guidelines were recommended for achieving this 
concept. Do not tell the pilot something that is good by any type of 
warning or cautionary device. Do not provide any type of signal if whatever 
is involved needs to be on during any major portion of the flight. Only 
annunciate when a system is off so that a light would be on only if there 
has been a failure or something has been turned off which normally should be 
on . 


In general, pilots favor the darkened cockpit concept wherein, during 
normal operation, no c/w or status lights are showing. There were some 
exceptions, however; a few pilots indicated they favored advisory lights 
which indicate that specific modes or conditions were in operation. There 
may be a real conflict here unless further investigation uncovers some simple 
guidelines for the use of blue, green, and white indicator lights. This is 
discussed more fully in a subsequent section. 

In general aviation aircraft, which are just beginning to encounter the 
addition of lights and c/w indications, many of the new lights dealt with 
navigation equipment. For example, green for all NAV Systems on, white for 
DME, and additional lights indicating auto-pilot and flight director modes. 

If the darkened concept is truly valid, some resolution of the extensive 
use of colored lights in these areas must be obtained. 

3 . '^ Lighting intensity and contrast are a serious problem in c/w 
systems . This appeared to be a greater problem in military aircraft having 
bubble-type canopies where extremely high contrast occurs due to direct 
sunlight at high altitude falling across a portion of the instrument panel. 
Cockpit lighting mockups have often been used to ensure acceptability of 
the cockpit design. The other problem which also can occur in transport 
aircraft is that of the high ambient light during a low visibility approach 
in fog. Comments were received which indicated that, with this high outside 
light level, interior cockpit lights often were not seen. These problems 
lead to other controversial questions, such as should the pilot have control 
over dimming of c/w lights in the cockpit. General aviation aircraft use a 
straightforward photo-electric cell to sense the ambient light and to adjust 
the intensity of cockpit lights automatically. At least one military 
fighter has adopted a rheostat control to allow the pilot to reduce lighting 
voltage and intensity even further than that normally provided by the 
reduction from 28 to 14 V in a two-step system. On the other hand, there is 
concern about giving pilots dimming control through a rheostat because, I 
presume, of the possibility of a light being inadvertently left dim when it 
should be bright. Design techniques to eliminate this possibility should be 
possible. One military contractor Indicates that current technology and 
design practices can and have eliminated the problem of achieving desired 
contrast by special treatment of the warning light surface. Some 
contractors felt that NASA should develop standards and techniques for 
determining brightness, contrast, and color compliance. 
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Guidelines Recommended by a Few Hut Overlooked by Most: General 


1- ' *Conf iguration warning is necessary for certain flight phases .'^ 
(i.e., takeoff and landing) This recommendation was prompted in one case 
because over a hundred takeoffs had been made in the wrong aircraft 
configuration due to a human factor in cockpit design and layout. In this 
case, if the electric actuator was used for setting the takeoff trim and the 
hydraulic trim wheel was in another position, the stabilizer would move to 
the hydraulic setting when hydraulic power subsequently came on, thereby 
destroying the pilot*s initial, supposedly proper, setting. During landing, 
the primary configuration warning required is the landing gear horn and 
light. Primary objections to these warnings relate to the fact that on many 
aircraft they come on any time the throttle is retarded, regardless of 
flight phase. This becomes a nuisance to the entire crew and a repeated 
distraction to the second officer. The solution mentioned most often is to 
restrict landing gear horn to appropriate speed and altitude ranges for 
landing and reduce unwanted occurrences at other times. A variable wing 
sweep aircraft was involved in several accidents because flaps were 
unattainable when selected unless the wings were completely unswept. A 
landing configuration warning was added that remedied the situation. 

2. '' Most urgent warnings should be related to the control involved ." 
Whenever the intent of this guideline can be met, it provides a direct cue 
which, in general, is unmistakable and enables the pilot to react 
instinctively. Of the warnings related to control involved, the stick-shaker 
is the one in most general use and the one most accepted as a good example 
of the application of tactile warning. In this case it is related directly 
to the control involved. One example of a situation wherein the stick-shaker 
was not favored was the F-111 in which a rudder-shaker was used instead. In 
this case the results were far from satisfactory as it was found that, due to 
improved handling characteristics, many pilots no longer flew with their 
feet on the rudders throughout much of the flight envelope. A second 
objection to the stick-shaker concept lies in the concept of vehicles with a 
stability and control augmentation system (SCAS) wherein a force transducer 
or pickoff is used to transmit the control input into *ah electronic control 
system. It is reasoned that spurious force inputs caused by a stick— shaker 
would be prohibitive in such a system. 

This guideline need not be restricted to tactile warnings but also 
suggests that urgent warnings should not be lost in the middle of a central 
or master warning panel. It also relates to the recommendations which were 
made to the effect that warnings related to pilot control, such as "pull 
up," should be located close to the instrument that the pilot is using, such 
as the ADI. As discussed elsewhere, some individuals even go so far as 
saying that the pilot-error type of warning must be integrated directly 
into the display being used. If it is related to flight path, then it should 
be evident in the flight-path control bar or indication; if in heading, then 
with the heading indication. 

Fire warning lights located in the extinguisher handle and system caution 
(inoperative/failed) lights located at the switch or lever which controls 
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that system are other examples. Where these are not readily visible, 
that is, where they are outside a 30° cone of vision, for example, some 
separate means of alerting the pilot is needed. The particular method used 
may also be a function of urgency. 

3 . requirement exists for additional warnings, that is, attitude , 

turbulence, wind shear, and collision avoidance . Several recommendations 
and/or predictions were made to this effect. Of primary concern at this 
point, however, is the simplification and design of an improved warning 
system utilizing guidelines that will enable the future addition of such 
warnings as their need becomes apparent and the technology is provided for 
their development. In other words, future cockpit design must allow for the 
incorporation, without adding to the distractions and confusion, of a few 
more important warnings. 

Two examples of other warnings which have been incorporated in response 
to specific requirements were provided. The Concorde supersonic transport 
incorporates Mach number and center of gravity indicators and warnings to 
alert and advise the pilot when approaching established limits too closely. 
One recent military aircraft uses an auditory signal of varying frequency to 
warn the pilot of an approaching yaw departure and incipient spin condition. 

Comparative warning systems are needed for certain critical instruments. 
Merely having redundancy in instruments or systems does not ensure prompt 
recognition and action. It was emphasized that a "comparator system" to 
provide an immediate alert to the pilot at the controls, is needed on many 
aircraft in order to reduce to an acceptable level the time required to 
recognize certain instrument failures. To a degree this has already been 
discussed under the paragraph concerned with reliability. A requirement now 
exists for attitude comparators on large multi-place military aircraft. 


Guidelines Recommended by a Few But Overlooked by Most: Auditory 

1. " Audio varnings should be used for pilot error situations only ." 

This may be related to the guideline that recommends that only system 
malfunctions should be shown on a master warning panel. This appears to be 
based on a desire to organize warnings in a more logical way. If one 
accepts other guidelines, which state that audio warnings should be reserved 
for emergency situations only, this one implies that pilot error warnings 
are the most serious. This, of course, is not the case, as minor human 
errors and misunderstandings occur through most operations but are recognized 
and corrected before leading to more serious problems. There is some 
question whether this guideline could be accepted and still not conflict 
with others which may be more important, such as reducing the total number of 
audio warnings or confining audio to emergency situations only. Some 
ambiguity exists also in regard, to whether voice warnings should be included 
in this guideline and their use thereby restricted. Taken at face value, 
this would eliminate the use of audio as an alerting device for any system 
malfunction. This is perhaps beyond what was envisioned, as immediate action 
warnings can arise from system failures. 
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In conclusion, there appears to be some basis for this guideline, but 
at present it is too broad and controversial to be accepted at face value. 

2 . radio override switch is needed to reduce interference by c/w 
systems . ” The question raised may be more complex than is immediately 
apparent. Is the conimunication to be overridden in order to distinguish the 
c/w signal, or voice warning? Or is it a cancellation of warning signals in 
order to maintain communications? It is possible that other than emergency 
audio warnings should not interfere with radio communication or should be 
cancellable but that emergency warnings must remain readable in spite of the 
communications system audio. From the overwhelming desire to reduce audio 
signals to a minimum, it might be assumed that only those remaining emergency 
ones need be heard above the normal radio communication. In cases where 
audio warning signals are piped through the communications speaker or 
headset, a slightly different situation exists than where only a separate 
electronic voice or audio warning speaker is used. 

3. " Voice warnings should be advisory in nature .” From experience 
with GPWS to date, there was some question as to whether a pilot will respond 
without delay to a "pull up" command. The majority of pilots apparently 
like the advisory "glide slope, glide slope," used in the GPWS, and object 

to the command "pull up, pull up." This led to the recommendation by some 
that a command like "pull up, pull up" should be preceded by a milder 
advisory. There were some indications that the large numbers of nuisance 
warnings which followed the rapid introduction of GPWS into service may have 
contributed to these objections. 

A subsequent study conducted by one GPWS manufacturer (ref, 1), 
however, provides a more optimistic picture, while still recognizing the 
human factors involved. A few examples are: "Data on 35 GPWS-equipped 

airplane incidents revealed that in more than 15 of them the pilot took 
positive corrective action while under instrument conditions." "GPWS has 
also detected 14 specific airport instrument approaches that were marginal." 
"Unwanted or nuisance warnings during instrument conditions have been almost 
non-existent." 

There was evidence during the survey that a warning command without a 
preparatory advisory was considered undesirable. However, there was an 
admission that emergency situations requiring immediate action may require 
a direct cue with a sense of urgency in order to reduce response time. There 
still appeared to be some reaction against such oral commands as "pull up, 
pull up," however. The use of a warning term like "terrain" in place of 
"pull up" has been suggested and may have merit. In any case, it was 
evident that the number of immediate action commands must be kept to a 
minimum. 
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Guidelines Recommended by a Few But Overlooked by Most: Visual 


1 . ^* The value of large, easy-to-read lettering or messages in a c/w 
system has been under-^emphasized . Concern was expressed by a number of 
those interviewed about the readability of some labeled lights, particularly 
under adverse conditions such as low contrast or turbulence. For example, a 
problem identified with many annunciator panels was that the lettering was 
often too small. 

Reference was made to one research program involving evaluation by a 
large number of airline pilots in which a particular prototype area 
navigation system was used as part of the test equipment. In this case, 
the lettering used was too small and resulted in the pilots having to shift 
position in order to distinguish the letters. This difficulty in reading 
became worse under turbulent conditions and contributed significantly to the 
time required to insert information and make changes in way points. It was 
concluded that these features resulted in significant adverse pilot opinion 
and contributed to a lack of acceptance and realization of some test 
objectives . 

The difficulty of setting exact limits on lettering size for messages 
is understandable because of the proliferation of annunciator and other 
lights on the panel and the limited panel space available. A problem of 
easy readability under normal lighting and non-turbulent conditions is 
usually the rule, and design is not always controlled by the worst case. 
Recommendations were obtained from one contractor who had established 
standards which ho felt ensured adequate visibility of labeled lights or 
messages. These were the use of 1/8-in letters at 28 in from the pilot’s 
eyes and the use of 5/8-in letters for the master caution and all red 
emergency lights. Military standards only specify that the maximum viewing 
distance be limited to 28 in. 

One interesting approach used in a NASA flight research program involved 
the use of a digital message line utilizing 1-in letters in the center of 
the forward instrument panel. The feeling obtained after discussing the use 
of such large, easy-to-read messages was that there could very well be some 
previous under-estimation of the benefits resulting from the speed with which 
visual, written or printed messages could be observed, understood, and acted 
upon. Obviously, the 1-in. letters may have gone beyond what is practical and 
necessary to permit rapid interpretation. It appeared highly possible that 
a reduction in interpretation and action time may result if the messages 
are provided in large, easy-to-read form, and further work to identify these 
limits may be desirable. Application in the most flexible form, of course, 
would be dependent on having a digital computer available for the generation 
of discrete messages. There was a significant preference established by 
those familiar with this type of digital message to the same written 
message on a CRT. Opinions were also expressed in the former program that 
a simple readout message appeared more acceptable to these pilots than a 
voice message would be. This may also be influenced by other factors, such 
as the use of command versus advisory voice messages discussed earlier. 
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These questions remain controversial. With the printing on annunciator 
lights often too small, there is an obvious need for a review of criteria in 
this area. 

2 . third color in addition to red and yellow is needed in caution/ 

warning systems . " 

^^ The use of all colored lights in the cockpit should be standardized 
There were a few recommendations that another color, in addition to red and 
yellow, was needed to identify a third category of warning. While red and 
yellow in general refer to immediate action and deferred action items, some 
feel the third color would indicate that something is not quite correct but 
that no action is required. Such items are also often referred to as 
advisories. Any clear-cut rules for the use of these colors was not made 
apparent during the survey, and they appeared to be left largely to the 
individual contractor. Even the Air Force admitted that the military 
specifications should be revised with respect to definitive application of 
the various colors — red, yellow, blue, green, and white. Some recommenda- 
tions were also made to the effect that a cautionary color in addition to 
aviation yellow should be introduced to differentiate between cautions that 
require no pilot action and those that require pilot action at a deferred 
time. 


A similar concept is recognized by the SAE S-7 Committee, which is 
studying the need for a third level of warning as well as other colored 
lights in the cockpit, whether or not they relate directly to a warning 
system. Such a possible fourth level has an effect on the "darkened 
cockpit" concept discussed earlier. 

Heretofore, colors like blue, green, and white have been used for 
advisories or to signify status of a system. This application appears to be 
growing, particularly with modular navigation and display units. It is 
apparent that both the civil and military favor some resolution of this 
problem, and concerted, coordinated action is needed to resolve these 
definitions and to identify the colors so used. Cr^iteria should therefore 
be redefined for each of the colored lights used in the cockpit and should 
also apply to the use of light emitting diodes (LED). It was pointed out 
that technology is only now developing to provide colors other than red for 
LED's. 

In addition to the colors already mentioned, there was a recommendation 
that, in taking a new look at the use of colors, attention should be 
directed to the use of combinations of red and blue as now used on many 
public safety vehicles. It was pointed out that an unidentified study at 
the University of Southern California indicated this to be an intolerable 
combination, one which is extremely annoying and therefore quite attention- 
getting, A second reason for considering this combination was the fact that 
many people are color-blind to red. 

A limited number of red and yellow warning lights are used in general 
aviation aircraft. Colored lights have been considered ineffective for stall 
warning and therefore are being eliminated from these aircraft. An audible 
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stall warning has been considered much more effective than a visual one. 
Cessna has found it necessary to add two more colors in order to 
differentiate the wheels up and down positions on a float plane. Here the 
normal red and green lights were insufficient because of the various 
combinations possible; therefore a blue light for gear-up and a brown-amber 
light for gear-down were added to identify when floats are installed. Most 
of the lights being added to these cockpits appear to be associated with 
informat ion-type advisories related to navigation and system operation. 


Guidelines with Mixed Reaction and Acceptance: General 

1. " An improved c/w system for retrofit is desirable .*' There was mixed 
reaction as to the desirability of a retrofit system, and after completing 
the survey, it was concluded that there was a misunderstanding as to what was 
deemed to be retrofit. Retrofit could be interpreted as placing a revised 
system in existing airplanes flying today, or it could be interpreted as 
placing the revised system in existing aircraft models which are yet to be 
produced, thereby enabling the revised system to be incorporated at the 
factory. It appears that the latter would be entirely feasible, where there 
are very limited possibilities of doing so in existing aircraft. The 
strongest possibility in this regard, however, was noted to lie with the 
electronic voice generator which, if found acceptable, could be designed so 
that a single unit, tied into the necessary sensors, would be all that was 
needed to be added to the cockpit. Many of the questions discussed under 
voice warning would have to be resolved before such a system would be 
accepted. This potential for retrofit of a single electronic voice generator 
was considered encouraging enough to warrant considerable effort being 
expended to resolve these questions. 

2. " Reverse the trend toward teaching pilots basic systems only (BSO) , 
that is, procedures only without a thorough knowledge of the aircraft 
systems . " This is really a question for airline training personnel, but it 
also involves basic concepts of aircraft systems design. It is probably not 
a subject for other than limited discussion here except that it was felt by 
some to foster a "dependence on warning devices" rather than on basic 
knowledge. It is of interest here because the increasing complexity of 
aircraft systems and cockpit information and controls are related to the c/w 
system used. The trend toward the use of on-board computers, electronic 
displays, and more automatic systems was also recognized by most of those 
interviewed. This problem will undoubtedly continue as an evolutionary 
development in which new designs and more automation must be reflected in 
revised training procedures, but a degree of iteration and swinging of the 
pendulum will probably continue. Whether basic systems only (BSO) aggravates 
or reduces the problem of c/w system design is beyond the scope of this 
paper. It is mentioned here because there were strong feelings among a 
number of those interviewed that this trend must be reversed and a return 
made to providing flight crews with a better understanding of their aircraft 
systems so that the handling of c/w indications does not become primarily a 
conditioned reflex. 
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Guidelines with Mixed Reaction and Acceptance; Visual 


1* '' A master warning light should be used. " We first need to dis- 

tinguish whether a master red warning light or a master-yellow caution light 
is being discussed. A master caution light is generally favored because of 
the fact that annunciator panels quite often are located in positions that are 
out of the pilot's direct visual cone. A single yellow light to serve the 
alerting purpose for cautionary indications is usually needed and is generally 
accepted for the cases where the annunciator panel is remotely located. 

Examples were also found where a red central warning light was needed 
because all red warnings were located in a separate panel to the side of the 
main instrument panel. Their use also may be dependent on whether red 
warnings are grouped near the ADI and other flight instruments, in a panel 
separate from yellow caution lights, or whether they are mixed into the same 
central panel. Again, examples of each exist. If a trend could be 
determined, it would be in line with the earlier guideline to reduce the 
total number of red warnings to less than 10, thereby making it possible to 
separate them from the main cautionary annunciator panel. The need, however, 
by nature of red warning indications (immediate action) is to attempt to 
group them as close to the ADI as is physically possible. The result there- 
fore is to cluster one master cautionary light with a variety of red warnings 
in this area. 

Some recommendations were specific in stating that master warning panels 
and master warning lights ought to be replaced with a single alerting device. 
This device would be aural, visual, or both, and the c/w information 
displayed on a CRT on a priority basis, supplemented by written instruction 
to the pilot as to action to be taken. Other preferences were expressed by 
some for separate warning and caution panels. 

2 . "The annunciator panel should display only system malfunctions and 
not pilot errors ." A similar recommendation was that there should be 
separation between cautions- and warnings resulting from human failure and 
those resulting from system failures and that the crew should be able to 
distinguish clearly between them. There was no clear-cut method for 
accomplishing this, however. Some felt that audio warnings should be reserved 
for crew failure situations. 

Another recommendation was that a master warning system should be 
confined to system malfunctions only. In such a system, a master warning 
light, centrally located for a pilot, alerts him to a c/w indication else- 
where. Several reasons for this interpretation were provided. First, any 
Information related to aircraft control and navigation should be associated 
with the instruments being used for this purpose. It was emphasized that the 
use of isolated warning lights that are not directly associated with the 
control element or information affected by crew error or deviation cause 
distraction and introduce delays. One example of associating the warning 
information with the instrument being used was the case where the radar 
altimeter bug could be set to give a fly-up command when reaching decision 
height. This eliminated any reliance on audio tones or lights. Another 
example of this type of warning is the use of the flight path symbol in a 
terrain-following display which, in case of failure or excessive deviation 
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below glide path, automatically gives the fly-up signal. The particular 
system in question also employs an adjoining red light affording a 
comparison between the two procedures. The former was clearly preferred, 
while the latter was considered distracting. 

Carried to its ultimate conclusion, the desire to separate system 
malfunctions from pilot errors may be based on the desirability of obtaining 
significant improvements to the basic flight display of information so that 
the pilot has no need for auxiliary warning devices. Military experience 
with integrated displays (large head-up) employing integration of all 
information required for the particular task, including airspeed, altitude, 
flight path angle, velocity vector, and deviation information, has shown 
this is what is really needed. Until such improved displays are available, 
the opinion was expressed that we would continue to have conflict between 
providing sufficient warning and alerting devices and total distraction of 
the pilot from his primary task. 

With a computer capability, it is possible to provide a written message 
pertaining to the warning in the integrated display, regardless of whether 
head-up or head-dowii. This was considered a desirable solution and would, of 
course, be applicable to any type of electronic attitude instrument. 

3. ^* Flashing lights should be used in c/w systems ." 

** Flashing lights are excellent attention-getters but are seriously 
distracting . '* There was general agreement that flashing lights are, at the 
same time, excellent attention getters but seriously distracting. There was 
no agreement on how the undesirable characteristics could be minimized. For 
this reason, they have been received with mixed reaction. The use of 
flashing lights has also met with mixed reaction due to their excellent 
visual alerting capability, but this is modified by the tendency of pilots 
to cancel them immediately. The Civil Aviation Authority (CAA) of the United 
Kingdom favors the use of a flashing light for alerting of either a primary 
(red) or a secondary (yellow) warning. It is clear that the CAA prefers a 
flashing master warning light on both the captain’s and first officer’s 
panel. On the master warning system panel, the CAA also feels that flashing 
lights are desirable because they can continue to flash without becoming 
as annoying as an audio signal. Further, they can be detected by peripheral 
vision. The French, however, favor an audio tone for alerting, so on joint 
aircraft programs an audio is added for primary warning. The airlines 
object to using the same flashing concept for both red and amber lights, so 
the concept of neither red nor amber flashing on the master warning system 
panel has been adopted. The military does not favor the use of flashing 
lights, but numerous waivers have been granted for their use in specific 
cases. The Air Force feeling is that flashing lights are generally too 
distracting and tend to restrict their use. 

Additional work is obviously needed to determine if, when, or where 
flashing lights should be used. One solution suggested was to automatically 
limit the flashing phase, after which the light reverts to a steady mode. 

It has also been suggested that the use of intermittent flashing lights, 
which remain on for 10 sec and then go off automatically, on a central 
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warning panel could eliminate the need for the master caution or warning 
light. On one recent aircraft, which uses several red^labeled warning 
lights near the ADI, plus a master caution (yellow) relating to the 
annunciator panel located elsewhere, the master caution light flashes until 
it is pushed to cancel, whereas each red light, which is labeled, is steady. 

Note that in this case the pilot and copilot are each furnished with all of 
these. Some other manufacturers, including those in general aviation, also 
use the concept of the master caution coming on flashing with the ability to 
be punched out. The fault still remains lighted on the annunciator panel. 

The application on one GPWS was the initial use of a steady red light 
which changes to flashing later. This appears to follow the concept of 
providing an advisory first and then a command but is counter to the concept 
of flashing for alerting. 

One concern expressed about the use of flashing lights, at least in the 
spacecraft application, is that they are extremely costly in terms of 
software. They are specifically used in presentation of information on a CRT 
as a means of drawing attention to specific information in a complex display. 

Pilots observed that, when faced with a flashing light, there is a 
strong reaction to cancel the light even before they know what it means. 

Whether a flashing light can at the same time be a good alerting device and 
yet not cause serious distraction is not clear because the effects appear to 
be somewhat contradictory. However, the ability to cancel a flashing light 
removes some of the distraction. One concept is that if a flashing light is 
adopted, it should be first in the flashing mode, thereby fulfilling its 
alerting function, and then go steady after initial action is taken, if this 
is what is desired. Some cases were found to exist wherein the light came 
on steady and then moved into a flashing mode. The option to cancel a 
flashing light was considered almost as important as the ability to cancel 
an audio signal. 

A second method of reducing the disturbing distraction of the continuous i 
flashing light is to automatically limit its flashing cycle. This, of 
course, assumes that it will have accomplished its alerting purpose during 
the initial cycle. When both an audio and visual alerting means are used, 
automatic control of the flashing cycle may be entirely adequate. Like audio 
warnings, it was emphasized that flashing lights need to be kept to a minimum. 
Several examples were noted wherein the concept of flashing first and steady 
second were violated. The flashing red light is sometimes used for an engine 
overheat condition with a steady red for a fire. It was noted that the 
latter is more serious and requires faster response. One solution on another 
aircraft was to separate the two lights and make them both steady, using 
other means to alert. 

On an L-1011 aircraft, operated by a foreign carrier, both the pilot 
and copilot have a master warning light which flashes until the fault is 
corrected. On one recent military aircraft, the master warning is a steady 
light, whereas the master caution flashes until it is pushed to cancel, 
leaving an annunciator light flashing on the central panel. One 
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recommendation was to use a flashing light on the central panel for 
alerting, with automatic cancellation after 10 sec; this process could 
eliminate the need for a master warning light. 

The flashing light is also sometimes used to indicate a transient 
condition, with a steady light referring to a steady condition. This led to 
the recommendation that the use of flashing lights needs to be standardized. 

That flash rate can be important was illustrated by military pilots 
interviewed who had experience with two different recent aircraft, each of 
which used a flashing light on the annunciator panel to indicate the most 
recent caution or warning. On one the flash rate was high, while on the 
other it was slower. The slower was strongly recommended as much easier to 
read. As information on the actual flash rates was not available, it must be 
assumed that the rates were within those specified by military standards. 

In any case, it suggests that improved criteria with respect to human factors 
in relation to the best range of flash rates appear to be needed. 


Guidelines with Mixed Reaction and Acceptance: Tactile 

1 • ** The use of tactile warnings should be extended in order to diversify 
the inputs to the pilot, ** This is a controversial area in which there 
exists a wide difference of opinion. There does appear to be some current 
effort, evolving partly under military sponsorship, to investigate extended 
use of tactile warnings in order to reduce reliance on audio and visual cues. 
There also exists the opposing viewpoint that, with the exception of the 
stlck-shaker, there is little interest in extending the use of tactile 
warnings. They are, in general, the least reliable because a pilot is not 
always able to discriminate their meaning. 

At least two companies, however, indicated an interest in the expanded 
use of tactile warnings and referred to work being conducted at Ohio State 
University with a control stick grip that makes use of tactile inputs to the 
pilot, and to Air Force Interest in the application of body pressure through 
segmented pneumatic seat and back cushions. 

One of the desirable aspects of the stick-shaker is the fact that it 
relates directly to the control involved In exercising corrective action. 

It might be expected, therefore, that tactile devices not related to the 
control involved or a clearly identifiable hazard may not find ready 
acceptance. 


ANALYSIS AND COMMENT 


In the preceding section, the discussion was almost exclusively confined 
to the observations made by respondents during the survey. In this section 
the information is summarized, along with comments and analysis of the writer, 
in order to develop the primary conclusions that have evolved from the study. 
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Less commentary is required for those guidelines and philosophies for which 
there was general acceptance, while more attention is devoted to contro- 
versial areas. Finally, a list of recommendations for investigation or 
research by NASA is provided, representing the opinions of both the 
respondents and the writer. 


General 

There appears to be validity in the conclusion that an excessive number 
of individual warning signals is presently provided in many transport and 
military aircraft. Fairly unanimous recommendations that immediate action 
warnings be reduced, that auditory signals be confined to four or five, and 
that the large numbers and complex arrangements of caution/warning lights 
be simplified and improved, certainly bear witness to this fact. The 
proliferation of warning sounds and lights, all competing for pilots’ atten- 
tion and prime instrument panel space in the pilots* central cone of vision 
is cited as compelling evidence of the need for reevaluation of the situation 
Considerable attention is therefore justified to studies of some of these 
basic questions in order to improve the guidelines for future designs and to 
correct design errors that appear to have been made in the past. A promising 
solution for reducing the large numbers of warnings which could occur at any 
given time lies in the use of some method for prioritizing and inhibiting 
warnings by phase of flight. 

The arguments for inhibiting and prioritizing warnings lie in the 
potential for reducing some crew overload conditions, and allowing faults or 
errors to be handled one at a time in a systematic manner according to a 
predetermined priority. Theoretically, this should lead to a decrease in 
reaction time and to a reduced likelihood of inappropriate corrective action. 

Risks are foreseen by some in not providing all warnings or caution 
indications immediately as they arise. One argument is that we are not 
intelligent enough to anticipate correctly the wisest priorities for all 
possible combinations of possible warnings- The counter argument may be 
that, if this is the case, we cannot expect a crew to resolve the same 
question any better within seconds while under the pressures of the 
operational situation. Another argument against prioritization lies in the 
fact that each type of aircraft is a unique design and no single system for 
prioritizing will be possible; it will vary among aircraft. 

Advanced technology can contribute effectively to the solution. 

Positive steps in this direction are reflected in current design trends 
involving increased automation, digital computers, and electronic displays 
in which information can be integrated and displayed in a variety of ways. 

It is possible also that the requirements for many of today’s warnings 
might be eliminated and a better solution provided by redesign of the cockpit 
information display system for normal operations; here, too, currently 
available computer and display technology could be utilized effectively. We 
must recognize, however, that such redesign will progress rather slowly in 
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actual application to transport aircraft and most likely will not be 
incorporated as retrofit in existing models. In addition, potential for 
improvement exists from better application of human factors doctrines in 
defining, not only what information is present, but also how it is best 
presented . 

Work in this area should deal, not only with the details of the c/w 
system, but also with other elements of the total operational situation. The 
failure of flight crews to heed obvious warnings signals, of which there are 
numerous examples, appears to be attributable, in significant measure, to the 
frequent high level of cockpit workload. In such an environment, the number 
of extra tasks that can be handled at a given time is reduced, and this 
affects adversely the crew^s ability to recognize and respond to information 
not relating directly to the cause of the high workload. Obviously, the 
factors contributing to high workload need to be identified and reduced or 
eliminated to enable c/w systems to retain their effectiveness. 

One additional factor that merits consideration as a contributor to the 
general problem is that of the fundamental training concept identified as 
"Basic Systems Only." A growing trend toward teaching "BSO" is believed by 
some to encourage pilots to become undesirably warning dependent. Obviously, 
the complexity of modern electro-mechanical^hydraulic systems makes it 
impractical for flight crews to attempt to understand and retain minute 
details of design, function, and performance. It is essential, however, 
that they have sufficient knowledge to operate such systems with safety and 
efficiency. How far the training of flight crews should go toward 
providing a limited understanding of these complex systems is a gray area 
which must be explored carefully. While there may be some basis for human 
factors studies of this problem, it is considered primarily a subject for 
manufacturers and airline training organizations and personnel and not one of 
high priority for NASA study. 


Reduce Number and Types of Warnings 

Reduction in the complexity of warning systems, as evidenced by the 
large numbers of both audio and visual indications, was the particular need 
most frequently mentioned. Emphasis here should be on reducing the number 
of immediate action warnings and the number of auditory signals. An 
effective improvement method that deserves immediate attention would be to 
inhibit unnecessary warnings on the basis of flight phase and to establish 
a prioritizing systems for those retained. It Is necessary, however, to 
accept the fact that the pilot must remain in a position to make final 
decisions regarding priorities; this poses the need to provide him with 
sufficient information on simultaneous problems to enable him to evaluate 
the overall situation properly. 

In addition, it is im^rtant that some means be provided for cancellation 
of warnings after they have served their primary purpose. Manual cancellation 
by the pilot is only one means that could be used. As one of the human 
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factors elements that could be studied in this area, consideration should be 
given to automatic cancellation or to intermittent warnings which allow 
periods of respite from the alerting distraction. 

One concept which appears to have merit is that, after cancellation of 
the central warning, whether red or amber, the fault information be retained 
on an auxiliary panel until corrected. The use of a flashing mode on the 
auxiliary panel until the fault is corrected has some merit in cases where 
numerous fault indicators remain lighted on a given flight. In the case of 
alpha-numeric message displays, where emphasis is on readability of only one 
or two messages, some method for storage and display of multiple faults must 
be devised. Possible methods include the use of a single symbol at the end 
of the message or a recall capability wherein all existing faults may be 
reactivated in the display, either sequentially as they occurred or on a 
prioritized basis. The method used may depend on whether the alpha-numeric 
message line or a CRT is employed. Determination of the best application of 
each should be made early in the investigation. 


Direct vs Indirect Cues 

The use of direct cues, that is, of those that provide information 
about corrective action, has been shown to reduce pilot reaction time by as 
much as 6 to 9 sec. While the value of reduced reaction time is obvious, 
there does not seem to be sufficient evidence at this time to establish that 
the means used for alerting the pilot can or should always be a direct cue. 
The potential value of direct cues over indirect ones, regardless of whether 
they are auditory, visual, or tactile, appears to be great, however. The 
direct cues which appear to provide the greatest potential are spoken or 
written messages, labeled lights, digital computer messages displayed on a 
CRT or LED type display, and the stick-shaker. 


There is considerable interest in and arguments for the use of audio 
tones only for alerting purposes and possibly to convey the proper level of 
warning. This would greatly simplify the use of warning sounds and enable 
more rapid standardization once an acceptable means for providing one or 
more direct cues is available in the cockpit. One argument against this 
procedure is that an alerting method that does not provide a direct cue, 
that is, unmistakable information on which the crew can act, results in 
unnecessary delay - 

Xn terms of possible retrofit and requiring the least redesign or space 
requirements on the instrument panel, the most promising methods available 
to improving the direct cues are voice and the alpha-numeric message 
display. 


Auditory 

There is general consensus that the use of voice cues through verbalized 
messages appears to be the direct auditory cue that provides the greatest 
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potential for payoff in future c/w systems. For this reason, it should be 
given the highest priority in any research and development efforts. In 
general, auditory signals have been considered, due to their compelling 
nature, to be reserved for pilot failure modes or for a few critical 
conditions or failures requiring immediate action or decision by the crew. 

It appears possible that the use of verbalized messages could be more 
widespread if the potential for conflict with other communications could be 
TTiinimized. In other words, voice messages could also provide an alternative 
to reading a visual message for lower level warnings which do not require 
inunediate action or can be recalled at a time when the crew is not overloaded. 
I'/hile much study has been given to the use of voice as to terminology, 
message content, and other characteristics, very little research has been 
conducted within the constraints of the operational situation, either simu- 
lated or real. Future work should not only reconsider questions of the 
message format and treatment of urgency level under simulated operational 
conditions, but should do so both in prioritized and non-priorit ized systems, 
incorporating multiple failures, and in conjunction with an integrated but 
separate visual warning system. 

There was a clear indication that pilots prefer advisory information to 
commands from voice warning. It is not clear yet how far this guideline 
should be carried in the design of voice warning systems, but such 
psychological aspects which affect individual response must be taken into 
account. If we accept the philosophy that the final decision is made by the 
on-board captain and crew, the issuance of "voice commands" by electro- 
mechanical means is a contrary philosophy. If commands are treated as 
advisory, with perhaps a level of urgency provided, the basic philosophy 
remains intact. A comparison may also be made with the flight director which 
is often referred to as a command instrument. If a pilot responds to every 
movement of a flight director, his attention becomes so intense on this one 
bit of information that his visual scan of other information is interrupted, 
his workload goes up, and he becomes a slave to this one source of informa- 
tion. If, on the other hand, he treats the flight director as a quickened 
bit of advisory information, he still achieves satisfactory performance 
without losing the scan pattern necessary to maintain situation awareness. 


Visual 

Although a well organized, systematized use of labeled lights has been 
recognized as reasonable and acceptable, there are characteristics even here 
which appear to warrant improvement. The question of readability under 
operational conditions, stemming from relative size of the lettering or 
cockpit lighting conditions, needs to be reconsidered in their design. 
Specific attention should be directed to the question of readability and 
clarity of visual messages under adverse lighting and vibration or turbulent 
conditions. While there is the potential of providing great detail in visual 
messages through computer displays, this must be balanced by the need for 
conciseness and readability. Specific new technology which should be 
evaluated includes the digital computer in conjunction with the concise 
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message readout versus the total CRT display. It is possible that both 
have a best application and may not be competitive in that sense. 

The use of flashing lights and flashing display elements is in need of 
a basic philosophy and guidelines for their use. The conflict regarding 
their use arises from the fact that they are at the same time an excellent 
means of attracting attention and that they may very easily become highly 
distracting. Some of the answer may lie in limiting the flashing period 
and using flashing lights only for alerting purposes, or perhaps as a means 
of identifying a level of urgency. On the other hand, the use of flashing 
yellow lights, as on an annunciator panel, appears to provide an excellent 
method of indicating that a fault remains uncorrected, while permitting the 
cancellation of a central warning. The use of a flashing mode in display 
elements to indicate unreliable information has apparent merit, but the 
concept must be reconciled with the use of flashing lights in warning systems 
The common parameter in each, however, appears to be that of drawing 
attention. 


Multiple Cues 

This leads to the question of multiple cues. In order to ensure that a 
warning is not overlooked, experience suggests that there is merit in 
providing multiple channels of information, that is, auditory, plus visual, 
or either of these plus tactile. While evidence to date indicates that 
multiple channels are required to ensure that pilots under high workload 
conditions have an increased capability for recognizing and responding to a 
warning, the possibility exists, with verbalized, digitized and printed CRT 
message concepts, for providing significant improvements in information 
content. These, plus improvements in the concepts for alerting the pilot, 
may make it possible to reduce such duplication and still achieve satis- 
factory results. Such a consideration should at least be a potential goal 
for any research arid development program. Recommendations have been made 
that auditory signals be reserved for alerting purposes and visual signals 
be used for action, but there does not appear to be sufficient evidence at 
this time to warrant such a broad concept. 


Uniform Standards and Guidelines 

The strong plea made by respondents for improved standards and guide- 
lines for warning systems is seen as an indication of their importance. It 
was evident that the military standards have been an important source of 
whatever standardization exists today. Their usefulness extends also to 
civil aviation where they can be used as guidelines without the effect of 
becoming requirements. 

In this regard, NASA should work closely with the FAA, and with Boeing, 
who is conducting studies under FAA sponsorship, to identify and provide 
support for updating those standards in need of updating. This will tend to 
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ensure that NASA research is conducted in a manner and within a time frame 
that will provide maximum application and hence the greatest return in terms 
of improvements in cockpit warning systems. 

Standards under development by the SAE S-7 Committee can provide an 
important step in this di,rection, largely because they will not have the 
effect of regulations, thereby enabling achievement of standardization by 
consensus and agreement without discouraging development. Despite the 
indicated value of the military standards, even the military agree they 
require updating and improving. During the survey only a few of the conflicts 
within the military standards were identified. An effort should be made to 
review these standards in detail, identifying all areas where change may be 
required, and a singular effort established to update them. Some form of 
coordination with the SAE S-7 Committee in this regard would seem desirable. 

One of the first places to begin standardization is in the terminology 
being used. This is currently a subject under study by the S-7 Committee. 

It is important for NASA to keep abreast of these developments so that 
current, revised, and approved terminology is used throughout any research 
and development program. 

Additional specific areas where new guidelines are needed in order to 
enhance future development are redefinition of categories or levels of 
warnings and the standardization of a few auditory signals which may be used 
either as alerting tones, with or without verbalized message, or as the few 
standard warnings that are universally applied to a few items, such as 
landing gear and fire. Improved guidelines are also needed to clarify the 
use of flashing lights in c/w systems, as is redefinition of the use of 
colored lights in order to retain the concept of a dark cockpit during 
normal operation. 


Reliability 

It has been shown that reliability is a significant determinant of the 
effectiveness of c/w systems. A c/w system is equally unreliable whether it 
malfunctions — to produce false fire warnings — or whether it is so 
designed or adjusted, with respect to limits, that it produces repeated 
nuisance warnings. An unfortunate illustration of the latter problem is 
afforded by the recent accelerated introduction of the GPWS into operational 
service without adequate preparation and evaluation. While remarkable 
progress has been made in achieving a high level of reliability with 
avionics systems, still more progress must be made if warning systems are to 
be accepted and used effectively by flight crews. 

Experimental studies involving new technology are, of course, very 
important, but they should not be conducted at the expense of comparative 
studies involving labeled lights and auditory signals, with and without 
verbalized warnings. Improved guidelines and recommendations for the 
application of these systems may« be significant in improving current cockpits. 
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Evaluation Procedures 


Perhaps the greatest deficiency in the past implementation of cockpit 
warning system has been a lack of objective evaluation under realistic 
operational conditions. Such an evaluation can be accomplished most 
expeditiously by using current advanced training simulators and full mission 
simulation techniques. Such programs are expensive, require a broad effort 
to implement, and require extensive data recording and reduction efforts. 

For these reasons, such full mission evaluation studies should probably be 
confined to the final evaluation phase of either new designs or comparative 
studies. They should be preceded by systematic experimental studies, 
beginning with simple laboratory experiments to separate the large number of 
variables or concepts possible and to understand the important differences 
between alerting and informing for decision. 

This focuses attention on the desirability of looking at c/w systems as 
very general in nature, incorporating not only the immediate action warning, 
but also the deferred or nonaction transfer of information to the pilot and 
crew, such as caution, advisories, and status indicators. These, then, 
relate to categories or levels of warnings based upon criticality or response 
required. Another aspect of importance appears to be the separation of the 
alerting function from the information part of the warning message. There 
could be value in considering a warning in terms of alerting, informing of 
the problem, and information relative to decision and action. It is 
suggested that the requirements of these three phases may differ. Is the use 
of a direct cue, for example, expected to accomplish all three? Or is 
alerting a separate function which might more clearly provide an alert if 
separated from the need to inform and generation of decision and action. 
Perhaps further experimental studies will be required to provide answers to 
these questions and to the desirability of adhering to a current military 
standards that require any warning to be preceded by an alerting signal or 
tone. Military standards also explicitly require a nonverbal signal to 
precede a verbal warning. 

Studies involving basic human factors and specific warning systems 
should consider those standards already outlined in the Mil. Stds. All and 
1A72 for specific confirmation or rejection of existing information. 

The next phase should be part-task simulation studies which involve 
combinations of the basic elements of cockpit warning systems for which a 
more refined comparison or evaluation is required. These will provide 
insight into the secondary advantages or disadvantages of specific elements 
but would not be expected to provide valid reaction times nor comprehension 
under realistic or high workload conditions. 

The fundamental performance measure in investigations of c/w systems is 
the pilot reaction time for proper response; this obviously involves 
considerations of comprehensibility and proper interpretation. Research 
data appear to be needed in order to define more authentic and realistic 
reaction times to warnings, particularly in the context of the total 
operational environment. A controversy over what should be accepted as 
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minimum response time may be related to the latter factor: 1 sec is 

recognized by the FAA, the CAA says 2 sec, and various pilot groups say it 
should be more like 6 to 7 sec. This question obviously warrants study. 

A review of the problems and recommendations obtained during this 
survey should provide a basis for research and development progressing from 
(1) specific elementary experiments to (2) part-task simulations involving 
more complete systems, and (3) a full mission simulation study involving 
two or more basic systems composed of subsystems selected from the results 
of (2). This is illustrated in table 4. 
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TABLE 4.- EXAMPLE OF SYSTEMATIC INVESTIGATION OF COCKPIT 
WARNING SYSTEMS CONCEPTS 


Specific laboratory 
experinients or studies 
involving c/w systems 
elements 

Part-task simulation 
studies involving 
c/w subsystems with 
important cockpit 
environment 

Full mission simulation 
studies to evaluate 
and compare complete 
c/w systems composed 
of most promising 
subsystems 

What are 5 best ^ 

auditory signals? 

Should auditory signals 

Part Task 

Simulation I 


be confined to 
immediate action? 

Scenario designed 


Should auditory signals, 

to evaluate 
experimental 

^ Full Mission 

including voice, incor- 

results 

Simulation 

porate urgency? How? 

> 


Scenario designed 



► to evaluate tenta- 



tive conclusions 

Should alerting signal 


from Part Task Simu- 

be used with voice? 

Part Task 

lator Evaluations I 


Simulation II 

and II and additional 

Should alerting signal. 


scenario content in 

if used, be continuous, 

Scenario designed 

footnote . 

push to cancel, or 

to evaluate 


auto-cancel ? 

experimental 


1 

What standards are de- 1 

results ^ 

► 


sired for voice warning 

- 


message context? 

What are best methods 
for indicating multiple 
warnings and prioritizing 
for single voice 
generators? 




Note: Also included in full mission scenarios would be results from similar 

systematic investigation of labeled lights, visual message display, 
CRT applications, etc. 
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RECOMMENDATIONS 


Each visit was concluded with a request for specific recommendations 
for investigation or research pertaining to warning and alerting. In this 
section are listed the primary recommendations received, some of which were 
specifically designated for NASA action. 


General 

1. NASA simulation data and support is needed on c/w systems. 

• NASA should be concerned with concepts only and not specific de- 
signs . 

• NASA effort should concentrate on providing data to guide and 
assist industry in the design of c/w systems. 

2. The need for new equipment pertaining to c/w systems should be 
confirmed by ensuring that it will accomplish its desired task 
before making it a requirement. 

3. NASA should provide objective evaluation of cockpit systems, 
including c/w, and should utilize full mission simulation in 
comparative testing of warning system concepts. 

4. NASA should provide basic human factors data for use by industry 
(e.g., realistic reaction times, acceleration thresholds, types of 
alarms, etc.). This should include determination of what 
functions a flashing light, bell, audio tones, and voice are best 
used for. It should also include an evaluation of methods for 
indicating the level of warning. 

5. The adequacy of methods by which pilots and copilots monitor each 
other should be determined. The relative merits of automatic 
monitoring and pilot monitoring should be determined. 

6. Methods for cancelling both red and yellow master warning lights 
and warning tone or voice messages should be evaluated. 

Auditory 

1. Develop and evaluate the use of voice warning in c/w systems, 
including the use of a single voice generator 

2. Conduct evaluations of voice warning in conjunction with a visual 
master warning panel system 
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3. Develop and evaluate a standardized voice warning system for 
inclusion in all aircraft; supplement this with a master c/w 
panel system as on the B-1 

4. Evaluate the use of modulated versus flat tone voice warning as 
well as methods for conveying a sense of urgency 

5. Investigate the removal of the annunciator panel from prime panel 
space by supplementing with voice 

6. Develop and evaluate standardized tones and guidelines for landing 
gear warning 

7 . Determine and evaluate a single master warning sound to alert for 
all critical items 

8. Determine five best audio sounds for basic standard auditory signals 

• Determine if these auditory signals should be limited to 
potentially catastrophic or most urgent situations only 

• Determine if these auditory signals should be reserved for 
crew failure only 


Visual 

1. Criteria should be redefined for what constitutes red, yellow, 
blue, green, or white light and for the use of LED colored lights. 

2. Investigate whether a third color is needed in addition to red and 
yellow for warnings and cautions. 

3. Brightness, contrast, and visibility of visual c/w elements should 
be investigated under a variety of cockpit lighting and vibration 
conditions. 

4. Visual warning systems need to be evaluated under conditions of high 
ambient light due to fog in low visibility approaches. 

5. Evaluate techniques for determining brightness, contrast, and color 
compliance with specif ications - 

6. Investigate the use of flashing lights and determine the best 
application in a c/w system. Resolve existing conflicts in their 
use. 

7. Investigate the use of a condensed alpha-numeric word display in 
conjunction with voice warning and contrast with a CRT display 
plus voice. 
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8. Investigate best ways of displaying c/w information on a CRT. 

• For systems information and checklists 

• For automated checklists 

• Other applications 

9. CRT use should be investigated for display of warning information 
on a priority basis, with written instructions on action to be 
taken. 

Tactile 

1. More tactile warnings should be developed and evaluated. 

Multiple Cues 

1. NASA should investigate and demonstrate the complementary use of 
voice warning and alerting with a visual system incorporating first, 
an alpha-numeric message readout, then with a standard labeled light 
system, then with a CRT display. 

2. A need exists for development of the logic for prioritizing c/w 
information and for redundancy requirements. Investigate and 
resolve methods for prioritizing messages. 

3. kinds and number of warnings that a pilot can prioritize should be 
determined, as should information on how pilots do prioritize 
information received. 

4. NASA should conduct studies to enable a change in Federal Aviation 
Rules against prioritizing of c/w signals. 

5. Methods should be developed and evaluated for incorporating or 
relating the control involved with each urgent warning. 

6. The use of auditory signals, including voice, should be evaluated 
against a visual system to determine effectiveness or deficiencies 
under conditions of high workload. 

Uniform Standards and Guidelines 

Mil. Stds. 1472 and 411 need revising, with emphasis on rationale 
behind them. 

2. Standards and guidelines for c/w system design are needed for 
transport aircraft. A needed revision of SAE 450C, “Visual, 

Audible and Tactile Signals on the Flight Deck,” is in progress. 
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New Warning Systems 


1. The need for attitude warning should be determined. 

2. A computerized system is needed which makes the potential of 
collision evident only when there is high probability of collision. 

3. A method is needed for integrating the GPWS or other add-on c/w 
items such as turbulence, wind shear, collision potential, or 
attitude into a central warning system. 
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BIBLIOGRAPHY 


This report is based primarily upon the information obtained during 
visits to 22 aviation organizations. The result of these visits were 
summarized in separate trip reports which comprise the primary source 
material for this report- (Items 1-22) While much additional work has been 
done on the subject of c/w systems and a bibliography of additional reference 
material was developed, it was considered too detailed for inclusion with 
this report and not pertinent to the specific objective of obtaining first- 
hand information from the industry. 

Trip Reports Relative to Aircraft Warning and Alerting Systems; 

1. Visit to Boeing Airplane Co., Seattle, Washington, October 18, 1975. 

2. Visit to VFW Fokker, Schiphol Airport, Amsterdam, The Netherlands, 
November 6, 7, 1*975. 

3. Visit to Aerospatiale, Toulouse, France, November 14, 1975. 

4. Visit to the Civil Aviation Authority, Redhill, England, 

November 20, 1975. 

5. Visit to the British Aircraft Corporation, Flight Test Department, 
Fairford, England, November 21, 1975.. 

6. Visit to Rockwell International, Los Angeles International Airport, 
El Segundo, California, March 3, 1976. 

7. Visit to Lockheed California Company, Burbank, California, 

March 4, 197B, 

8. Visit to McDonnell-Douglas , Long Beach, California, March 4, 1976. 

9. Visit to McDonnell-Douglas , St. Louis, Missouri, March 23, 1976. 

10. Visit to Wright-Patterson Air Force Base, Dayton, Ohio, 

March 24, 25, 1976. 

11. Visit to the Navy Safety Center, NAS, Norfolk, Virginia, 

April 5, 1976. 

12. Visit to meeting of the SAE S-7 Committee, Washington Hilton Hotel, 
Washington., D. C., April 6, 7, 1976. 

13. Visit to FAA Headquarters, Washington, D. C,, April 8, 1976. 

14. Visit to the Grumman Aerospace Company, New York, April 9, 1976, • 

15. ’Visit to Joint Test Force for the B-1, Edwards Air Force Base, 
California, May 13, 1976. 
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16. Visit to USAF Directorate of Aerospace Safety, Norton Air Force 
Base, California, May 14, 1976. 

17. Visit to Sperry Flight Systems, Phoenix, Arizona, May 17, 1976. 

18. Visit to Cessna Aircraft Co., Wichita, Kansas, May 25, 1976. 

19. Visit to Beech Aircraft Co., Wichita, Kansas, May 25, 1976. 

20. Visit to General Dynamics Corporation, Fort Worth, Texas, 

May 26, 1976. 


21. Visit to NASA, Johnson Spacecraft Center, Houston, Texas, 

May 27, 1976. 

22. Visit to NASA, Ames Research Center, Moffett Field, California, 
June 1976. 
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